Electronic device and control method thereof

ABSTRACT

An electronic device comprises: an image sensor; an input unit used for inputting an instruction that accompanies switching of an actuation method of the image sensor; and a controller that switches the actuation method of the image sensor using either of a first switching method and a second switching method based on a predetermined condition according to an input of the instruction. In the first switching method, the actuation method is switched via a standby state of the image sensor according to the input of the instruction, and in the second switching method, the actuation method is switched at a timing of a start of a next frame immediately after the input of the instruction while actuating the image sensor at a predetermined frame rate.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an electronic device and a controlmethod thereof, and more particularly to a technique for switchingactuation methods of an image sensor in an electronic device.

Description of the Related Art

There are digital cameras and electronic devices with camera functions(hereinafter collectively referred to as “image capturing apparatuses”)which can operate in a live view (LV) mode, moving image shooting mode,and still image shooting mode.

The LV mode is a shooting mode in which images obtained by developing animage signal output from an image sensor are displayed on a displaydevice at a constant frame rate.

The moving image shooting mode is a shooting mode in which a movingimage with a constant frame rate obtained by performing developmentprocessing on an image signal output from the image sensor is recordedon a recording medium. For moving image data recorded in the movingimage shooting mode, there are recording sizes such as high-definition4K and normal Full High Definition (FHD). Also, there are 60 fps, 30fps, etc. as the frame rate for recording.

A still image shooting mode is a shooting mode in which a still imageobtained by performing development processing on an image signal outputfrom an image sensor is recorded on a recording medium. In the stillimage shooting mode, the recording size of the still image can beselected. For example, if the image sensor has 24M pixels, the recordingsize may be 24M-pixel-recording, 12M-pixel-recording, and so forth.

Since the resolutions of image data is usually different between an LVimage, a moving image, and a still image, the actuation methods of theimage sensor are also different. Therefore, when changing the shootingmode, the actuation method of the image sensor is changed accordingly.

For example, when the LV mode is switched to the moving image shootingmode, the output of image data from the image sensor being actuated inthe LV mode is temporarily stopped, and after switching the actuationmethod of the image sensor to that of the moving image shooting mode,the output of the image data from the image sensor is restarted.Therefore, an image of the previous frame may be displayed on thedisplay device, or a fixed color image such as a black image may bedisplayed, because the image data does not exist.

Japanese Patent Laid-Open No. 2015-186234 discloses a structure in whichimage data is stored in a memory within an image sensor, and an image isread out from the memory, and discloses that, upon switching theactuation method, storage of image data that may become a defectiveframe is stopped, and image data stored before the switching of theactuation method is read out.

Further, Japanese Patent Laid-Open No. 2020-184699 discloses a method,when still image shooting is instructed during moving image shooting, ofavoiding a state in which the displayed image is stopped (frame stop) byperforming the still image shooting during a blanking period of themoving image shooting.

In the method described in Japanese Patent Application Laid-Open No.2015-186234, image data stored before switching the actuation method isoutput from the image sensor, so the same image is kept displayed on thedisplay device of the electronic device for two or more frames, and thedisplayed image becomes still.

Further, in the case of Japanese Patent Laid-Open No. 2020-184699, whenthe frame rate of the moving image shooting is low, the responsivenessfrom the instruction of the still image shooting to the start of thestill image shooting is poor.

SUMMARY OF THE INVENTION

The present invention has been made in consideration of the abovesituation, and improves the responsiveness in switching betweenactuation methods of an image sensor.

According to the present invention, provided is an electronic devicecomprising: an image sensor; and one or more processors and/or circuitrywhich function as: an input unit used for inputting an instruction thataccompanies switching of an actuation method of the image sensor; and acontroller that switches the actuation method of the image sensor usingeither of a first switching method and a second switching method basedon a predetermined condition according to an input of the instruction,wherein, in the first switching method, the actuation method is switchedvia a standby state of the image sensor according to the input of theinstruction, and wherein, in the second switching method, the actuationmethod is switched at a timing of a start of a next frame immediatelyafter the input of the instruction while actuating the image sensor at apredetermined frame rate.

Further, according to the present invention, provided is a controlmethod of an electronic device comprising: switching, according to aninput of an instruction that accompanies switching of an actuationmethod of an image sensor, the actuation method of the image sensorusing either of a first switching method and a second switching methodbased on a predetermined condition, wherein, in the first switchingmethod, the actuation method is switched via a standby state of theimage sensor according to the input of the instruction, and wherein, inthe second switching method, the actuation method is switched at atiming of a start of a next frame immediately after the input of theinstruction while actuating the image sensor at a predetermined framerate.

Furthermore, according to the present invention, provided is anon-transitory computer-readable storage medium, the storage mediumstoring a program that is executable by the computer, wherein theprogram includes program code for causing the computer to perform acontrol method of an electronic device comprising: switching, accordingto an input of an instruction that accompanies switching of an actuationmethod of an image sensor, the actuation method of the image sensorusing either of a first switching method and a second switching methodbased on a predetermined condition, wherein, in the first switchingmethod, the actuation method is switched via a standby state of theimage sensor according to the input of the instruction, and wherein, inthe second switching method, the actuation method is switched at atiming of a start of a next frame immediately after the input of theinstruction while actuating the image sensor at a predetermined framerate.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are incorporated in and constitute apart of the specification, illustrate embodiments of the invention, andtogether with the description, serve to explain the principles of theinvention.

FIG. 1 is a block diagram showing a schematic configuration of an imagesensor according to an embodiment of the present invention.

FIG. 2 is a diagram showing a pixel array of the image sensor accordingto the embodiment.

FIG. 3 is a block diagram showing a schematic configuration of an imagecapturing apparatus according to the embodiment.

FIGS. 4A and 4B are timing charts showing a switching operation of afirst actuation switching method according to a first embodiment.

FIGS. 5A and 5B are timing charts showing a switching operation of asecond actuation switching method according to the first embodiment in acase where a frame rate of LV image shooting is high.

FIG. 6 is a timing chart showing a switching operation of the secondactuation switching method according to the first embodiment in a casewhere the frame rate of LV image shooting is low.

FIG. 7 is a flowchart showing determination processing of an actuationswitching method according to the first embodiment.

FIG. 8 is a flowchart showing determination processing of an actuationswitching method according to a modification of the first embodiment.

FIG. 9 is a timing chart showing a switching operation in a case where aframe rate of LV image shooting is low according to the modification ofthe first embodiment;

FIG. 10 is a timing chart showing a switching operation of the firstactuation switching method according to a second embodiment in a casewhere the frame rate of LV image shooting is low.

FIG. 11 is a timing chart showing a switching operation of the secondactuation switching method according to the second embodiment in a casewhere the frame rate of LV image shooting is low.

FIG. 12 is a flowchart showing determination processing of an actuationswitching method according to a third embodiment.

DESCRIPTION OF THE EMBODIMENTS

Hereinafter, embodiments will be described in detail with reference tothe attached drawings. Note, the following embodiments are not intendedto limit the scope of the claimed invention, and limitation is not madean invention that requires a combination of all features described inthe embodiments. Two or more of the multiple features described in theembodiments may be combined as appropriate. Furthermore, the samereference numerals are given to the same or similar configurations, andredundant description thereof is omitted.

First Embodiment

Configuration of Apparatus

FIG. 1 is a block diagram showing a schematic configuration of an imagesensor according to a first embodiment of the present invention.

An image sensor 306 has a plurality of pixels 101 arranged in a matrix,and is connected to transfer signal lines 103, reset signal lines 104,and row selection signal lines 105 in the horizontal direction (rowdirection). Further, the pixels 101 in each row of the image sensor 306are connected to any of a plurality of vertical output lines 102provided for each column in the vertical direction (column direction).With this configuration, signals are read out from the pixels in unitsof a plurality of rows.

Further, the image sensor 306 includes an actuation circuit comprised ofcolumn ADC blocks 111, a row scanning circuit 112, column scanningcircuits 113, and a timing control circuit 114, a changeover switch 116,and a parallel serial (P/S) conversion unit 117. The changeover switch116 switches between two image signals output via horizontal signallines 115-a and 115-b, and outputs the selected image signal.

The timing control circuit 114 controls actuation timing of the imagesensor in synchronization with a vertical synchronization signal (VDsignal) 118, a horizontal synchronization signal 0 (HD0 signal) 119, anda horizontal synchronization signal 1 (HD1 signal) 120, all of which areinput from an overall control calculation unit 309 which will bedescribed later. An SIO communication signal 121 is input to the imagesensor 306.

The image signal is subjected to parallel/serial conversion in the P/Sconversion unit 117 in accordance with the timing signal from the timingcontrol circuit 114 and sent to the outside of the image sensor 306. Asactuation method of the image sensor 306 for reading out an imagesignal, it is possible to select one of an actuation method of readingout an image signal from all pixels, an actuation method of reading outan image signal from pixels thinned out in the vertical direction by ⅓or ⅕, an actuation method of reading out an image signal from pixel rowsexcluding upper and lower portions in the vertical direction, and soforth.

The image sensor 306 is controlled between a standby state, readoutstate, and exposure state.

In the standby state, reading out of an image signal from the pixels 101and image data transfer to an image signal processing circuit 307, whichwill be described later, are not performed. In the readout state, imagesignals are read out from the pixels 101 and transferred to the imagesignal processing circuit 307. In the exposed state, the pixels 101 areexposed and charges are accumulated without being read out or withouttransferring image data to the image signal processing circuit 307.

FIG. 2 is a diagram schematically showing the pixel array of the imagesensor 306 covered with color filters 202. As an example, the colorfilters 202 are arranged in a Bayer pattern, and red (R) and green (Gr)color filters 202 are alternately provided on pixels in odd rows fromthe left, and green (Gb) and blue (B) color filters are alternatelyprovided on pixels in even rows from the left. Also, an on-chipmicrolens 201 is formed on the color filter 202.

FIG. 3 is a block diagram showing a schematic configuration of an imagecapturing apparatus according to the first embodiment which is providedwith the image sensor 306 having the above configuration.

In FIG. 3 , a lens actuation unit 302 actuates a lens unit 301 tocontrol the zooming, focusing, etc. of the lens unit 301. A mechanicalshutter 303 and a diaphragm 304 are controlled and actuated by ashutter/diaphragm actuator 305. An amount of light of an image of asubject to be incident through the lens unit 301 and the mechanicalshutter 303 is appropriately adjusted by the diaphragm 304, and theimage is formed on an imaging surface of the image sensor 306 by thelens unit 301.

The image of the subject formed on the imaging surface of the imagesensor 306 is photoelectrically converted by the pixels 101 andgain-adjusted, and a resultant analog signal is converted to a digitalsignal which is captured as signals of R, Gr, Gb and B, and then thedigital signal is sent to the image signal processing circuit 307.

An image signal processing circuit 307 performs predetermined arithmeticprocessing using the image signal output from the image sensor 306, andthe overall control calculation unit 309 performs exposure control andfocus adjustment control based on the obtained arithmetic results. As aresult, TTL (through-the-lens) AF (autofocus) processing, AE (automaticexposure) processing, and EF (automatic flash light emission) processingare performed. By lengthening or shortening the interval between pulsesof the VD signal 118 given to the image sensor 306 in AE processing, itis possible to change the frame rate.

Further, the image signal processing circuit 307 performs predeterminedarithmetic processing using the image signals output from the imagesensor 306, and also performs TTL AWB (auto white balance) processingbased on the obtained arithmetic result.

Further, the image signal processing circuit 307 performs defect pixelcorrection and shading processing on the image signal output from theimage sensor 306, and performs development processing. In thedevelopment processing, low-pass filter processing for reducing noise,sharpness correction for correcting blurring of the subject, contrastcorrection for adjusting the contrast of the image, false colorcorrection for correcting false colors, and the like are performed.

The image signal processed by the image signal processing circuit 307 isrecorded on a recording medium 312 via a recording medium controlinterface (I/F) unit 310 or displayed on a display unit 311 according toinstructions from the overall control calculation unit 309. The displayunit 311 includes an electronic view finder (EVF) and a liquid crystaldisplay (LCD). By sequentially displaying the image signal processed bythe image signal processing circuit 307 on the display unit 311 frame byframe, a live view (LV) image can be displayed.

The recording medium 312 is a removable storage medium such as asemiconductor memory, and the image signal recorded on the recordingmedium 312 can be read out via the recording medium control I/F unit310.

A first memory unit 308 is used to temporarily store image signals. Asecond memory unit 314 stores the calculation result of the overallcontrol calculation unit 309.

An external interface (I/F) unit 313 is an interface for communicatingwith an external computer or the like.

An operation unit 315 is used by the user to input various settings andinstructions, and includes a shutter release button for instructing thestart of still image shooting, a recording button for instructing thestart and end of moving image shooting, a menu button, a playback buttonfor checking the captured image, and so forth.

The shutter release button is a two-stage switch consisting of a switchSW1 and a switch SW2, and the switch SW1 is turned ON in the middle ofoperation (for example, half-pressing), and preparation for still imageshooting such as AF processing, AE processing, AWB processing, EFprocessing, and so on is instructed. Further, when the operation iscompleted (for example, full pressing), the switch SW2 is turned ON andstill image shooting is instructed.

When the recording button is pressed once, recording of a moving imageis instructed, and when the recording button is pressed during recordingof the moving image, end of recording the moving image is instructed.Note that the shutter release button and recording button are notlimited to these configurations, and any operation member that caninstruct the start and end of still image shooting and moving imageshooting may be used.

The overall control calculation unit 309 controls the entirety of theimage capturing apparatus and performs various calculations. The overallcontrol calculation unit 309 controls the entirety of the imagecapturing apparatus including an LV mode, still image shooting mode, andmoving image shooting mode according to the conditions set by the userusing the operation unit 315.

First Actuation Switching Method

FIGS. 4A and 4B are timing charts showing the switching operation of theimage sensor 306 by the first actuation switching method in the firstembodiment.

In the following description, the image sensor 306 is actuated by a lowimage quality actuation method in the LV mode, and the image sensor 306is actuated by a high image quality actuation method in the moving imageshooting mode. Further, it is assumed that the frame rate of the imagesensor 306 is 30 fps (33 ms) in the LV mode, and 30 fps (33 ms) in themoving image shooting mode. Furthermore, it is assumed that time takento read out the signal from the image sensor 306 in the LV mode ormoving image shooting mode is 2 ms. It is also assumed that the displayunit 311 operates at 60 fps in the first embodiment.

Furthermore, in the following description, since the contents of controlperformed in response to the SIO communication signal 121 are different,the SIO communication signal 121 is described as SIO (1) to (4)communications according to the contents of control in order todistinguish between them.

When the start of moving image recording is instructed from theoperation unit 315 during the LV mode, the overall control calculationunit 309 performs SIO(1) communication to send an actuation switchingrequest to the image sensor 306. In the SIO(1) communication, thesetting such that the image sensor 306 transits to the standby state atthe timing of the next pulse of the VD signal being input is performed.The timing at which this SIO(1) communication is performed changesdepending on at which timing in the blanking period the start of movingimage recording is instructed.

FIG. 4A is a diagram showing a case where a start of moving imagerecording is instructed during a setting prohibition period of 1 msbefore the input of a pulse of the VD signal. In this case, since theSIO(1) communication is performed immediately after the next pulse ofthe VD signal is generated, the reading of the signal from the imagesensor 306 corresponding to EXPOSURE (n+1) in FIG. 4A is stopped in themiddle, and thus an image that should be DISPLAY (n+1) cannot becaptured. As a result, an image DISPLAY (n), is continuously displayedon the display unit 311 for two or more frames, which causes aphenomenon in which the displayed image looks temporally stopped.

When the image sensor 306 enters the standby state according to theinput of the next pulse of the VD signal through the SIO(1)communication, the overall control calculation unit 309 performs theSIO(2) communication during the standby state. The SIO (2) communicationis to change the actuation method of the image sensor 306, send the VDsignal and the HD0 signal to the image sensor 306 in accordance with thechanged actuation method, set an exposure period via the reset signallines 104, and further set gains to column ADC blocks 111.

Next, the overall control calculation unit 309 issues the next pulse ofthe VD signal in synchronization with the display timing of the displayunit 311 (Sync synchronization period), the image sensor 306 is changedto the readout state, and EXPOSURE (n+2) by the high image qualityactuation is started. Since the sync synchronization period is between 3ms and 16 ms, it takes about 37 ms at the shortest and about 50 ms atthe longest from when the start of moving image recording is instructeduntil when the image (DISPLAY (n+2)) obtained by high image qualityactuation is displayed on the display unit 311.

FIG. 4B is a diagram showing a case where the start of moving imagerecording is instructed immediately after the blanking (BLK) periodafter completion of a readout operation has started. In this case, theSIO(1) communication is performed immediately after the end of thereadout operation. The overall control calculation unit 309 issues apulse of the VD signal when the SIO(1) communication for an actuationswitching request is completed, and shifts the image sensor 306 to thestandby state. Image data is not transferred from the image sensor 306during the standby state of the image sensor 306. Therefore, the imagedisplayed on the display unit 311 is not updated, and the displayedimage is temporarily frozen.

When the image sensor 306 becomes the standby state, the overall controlcalculation unit 309 performs the SIO(2) communication. In the SIO (2)communication, the actuation method of the image sensor 306 is changed,the VD signal and the HD0 signal are sent to the image sensor 306according to the changed actuation method, an exposure period is set viathe reset signal lines 104, and further, set gains to the column ADCblocks 111.

Next, the overall control calculation unit 309 issues the next pulse ofthe VD signal in synchronization with the display timing of the displayunit 311 (Sync synchronization period), the image sensor 306 is changedto the readout state, and EXPOSURE (n+2) by the high image qualityactuation is started.

However, since the period during which the image shown as DISPLAY (n+1)is displayed on the display unit 311 is extended by 16 ms, thesmoothness of the displayed image is lost and a phenomenon in which thedisplayed image looks temporally stopped occurs. In addition, since theSync synchronization period is 3 ms to 16 ms, it takes about 36 ms atthe shortest and about 49 ms at the longest since the start of movingimage recording is instructed until an image (DISPLAY (n+2)) obtained bythe high image quality actuation is displayed on the display unit 311.

Thus, in the first actuation switching method, a phenomenon in which thedisplayed image looks temporally stopped occurs, but the time taken fromwhen the start of moving image recording is instructed to when adisplayed image is changed falls within a range between about 36 ms andabout 50 ms.

Second Actuation Switching Method

FIGS. 5A and 5B are timing charts showing the switching operation by theseamless actuation switching method as a second actuation switchingmethod of the image sensor 306 in the first embodiment.

In FIGS. 5A and 5B as well, the image sensor 306 is actuated by the lowimage quality actuation method in the LV mode, and the image sensor 306is actuated by the high image quality actuation method in the movingimage shooting mode. Further, it is assumed that the frame rate of theimage sensor 306 is 30 fps (33 ms) in the LV mode, and 30 fps (33 ms) inthe moving image shooting mode. Furthermore, it is assumed that timetaken to read out the signal from the image sensor 306 in the LV mode ormoving image shooting mode is 2 ms. It is also assumed that the displayunit 311 operates at 60 fps in the first embodiment.

When the start of moving image recording is instructed from theoperation unit 315 during the LV mode, the overall control calculationunit 309 performs SIO(3) communication to send an actuation switchingrequest to the image sensor 306. In the SIO(3) communication, thereading of the signal from the image sensor 306 is set to be performedby the low image quality actuation method and the exposure is set to beperformed by the high image quality actuation method at the timing ofthe next pulse of the VD signal. The timing at which this SIO(3)communication is performed changes depending on at which timing in theblanking period the start of moving image recording is instructed.

FIG. 5A is a diagram showing a case where the start of moving imagerecording is instructed during a setting prohibition period of 1 msbefore the input of a pulse of the VD signal, and showing the case wherethe longest time will be taken to switch the image after the start ofmoving image recording is instructed. In this case, the SIO(3)communication is performed immediately after readout of signals startedin response to the next pulse of the VD signal is completed.

At this time, if the HD period (horizontal period) differs between thelow image quality actuation method and the high image quality actuationmethod, the HD0 signal is used in the low image quality actuationmethod, and the HD1 signal is used in the high image quality actuationmethod to control the actuation timing of the image sensor 306. Inaddition, the VD signal for the high image quality actuation method issent to the image sensor 306, an exposure period for the high imagequality actuation method is set via the reset signal lines 104, andgains for the low image quality actuation method are set in the columnADC blocks 111.

Next, when a pulse of the VD signal is generated after 33 ms, SIO(4)communication is performed. In the SIO(4) communication, the imagesensor 306 is controlled such that the high image quality actuationmethod will be set for readout and the high image quality actuationmethod will be set for exposure at the next pulse of the VD signal.Also, an exposure period for the high image quality actuation method isset via the reset signal lines 104, and gains for the high image qualityactuation method are set in the column ADC blocks 111.

In this case, it takes about 67 ms from when the start of moving imagerecording is instructed to when an image (DISPLAY (n+3)) obtained by thehigh quality actuation is displayed on the display unit 311.

FIG. 5B is a diagram showing a case where the start of moving imagerecording is instructed 1 ms before the input of a pulse of the VDsignal, and shows a case where it takes the shortest time since thestart of moving image recording is instructed until an image isswitched. Unlike FIG. 5A, the SIO (3) communication is performedimmediately after the start of moving image recording is instructed andthe SIO(4) communication is performed when a pulse of the VD signalhaving a period of 33 ms is generated.

In this case, it takes about 34 ms from when the start of moving imagerecording is instructed until when the display unit 311 displays animage (DISPLAY (n+3)) obtained by the high quality actuation.

In the case of the second actuation switching method shown in FIGS. 5Aand 5B, unlike the first actuation switching method shown in FIGS. 4Aand 4B, the image sensor 306 is controlled such that the exposure of theimage sensor 306 and the readout of a signal from the image sensor 306are not stopped without transitioning the image sensor 306 to thestandby state. For this reason, a phenomenon in which an image displayedon the display unit 311 looks temporally stopped does not occur, asshown by DISPLAY (n), DISPLAY (n+1), DISPLAY (n+2), and DISPLAY (n+3).Also, the time from when the start of moving image recording isinstructed to when a displayed image is switched falls within a rangebetween about 34 ms to about 67 ms, which is about the same range as inthe first actuation switching method. However, this range becomes longerwhen the frame rate before the switching is lower.

FIG. 6 is a timing chart showing a seamless actuation switching methodwhen the frame rate of the image sensor 306 is low.

Here, the image sensor 306 is actuated by the low image qualityactuation method in the LV mode, and the image sensor 306 is actuated bythe high image quality actuation method in the moving image shootingmode. Further, it is assumed that the frame rate of the image sensor 306is 15 fps (66 ms) in the LV mode, and 30 fps (33 ms) in the moving imageshooting mode. Furthermore, it is assumed that time taken to read outthe signal from the image sensor 306 in the LV mode or moving imageshooting mode is 2 ms.

In FIG. 6 , in a case where the start of moving image recording isinstructed at timing 1 in the setting prohibition period of 1 ms beforea pulse of the VD signal is input, it takes about 100 ms since the startof moving image recording is instructed until an image (DISPLAY (n+2))obtained by the high quality actuation is displayed on the display unit311, wherein the 100 ms consists of the setting inhibition period of 1ms, the 1 VD period of 66 ms, and the EXPOSURE (n+1) period of 33 ms.

On the other hand, if the start of moving image recording is instructedat timing 2 immediately before the setting prohibition period, it takesabout 34 ms since the start of moving image recording is instructeduntil the image (DISPLAY (n+2)) obtained by the high image qualityactuation is displayed on the display unit 311, wherein the 34 msconsists of 1 ms until the next pulse of the VD signal is generated andthe EXPOSURE (n+2) period of 33 ms. That is, a variation of 66 ms occursdepending on the timing at which the start of moving image recording isinstructed.

Thus, in the second actuation switching method, the time taken to switcha displayed image increases as the frame rate of the image sensor 306actuated by the low quality actuation method becomes lower.

Switching Control of Actuation Method

FIG. 7 is a flowchart showing determination processing of an actuationswitching method for suppressing variation in time since the start ofmoving image recording is instructed until the high quality actuationmethod is set in the first embodiment.

When a start of moving image recording is instructed, it is determinedin step S701 whether the current frame rate is high. Here, the currentframe rate is compared with a predetermined threshold value T1 (forexample, 30 fps), and if the current frame rate is equal to or higherthan the threshold value T1, it is determined that the frame rate ishigh.

If it is determined that the current frame rate is high, the processproceed to step S702 and it is determined that the second actuationswitching method is to be used to switch the actuation method. If it isdetermined that the current frame rate is not high, the process proceedsto step S703 and it is determined that the first actuation switchingmethod is to be used to switch the actuation method.

As described above, in the first actuation switching method, thedisplayed image momentarily stops, but the time taken to switch theactuation method falls within a range between about 36 ms at theshortest and about 50 ms at the longest. Therefore, if the current framerate is not a high frame rate, the first actuation switching method, notthe second actuation switching method, can suppress the variation intime taken to switch the actuation method.

On the other hand, if the current frame rate is a high frame rate, thesecond actuation switching method enables smoother image display thanthe first actuation switching method, and the variation in time taken toswitch the actuation method substantially falls within that of the firstactuation switching method.

As described above, according to the first embodiment, a variation intime taken to switch the actuation method can be suppressed whilemaintaining smoothness of a displayed image as much as possible at thetime of switching the actuation method when the LV mode is changed tothe moving image shooting mode.

Modification

As shown in FIG. 6 , the seamless actuation switching method, which isthe second actuation switching method, has the problem that thevariation in time taken to switch the actuation method increases as theframe rate becomes low. To solve this problem, in this modification,whether to use the first actuation switching method or the secondactuation switching method is determined further based on the timing atwhich the start of moving image recording is instructed, therebysuppressing the variation in time taken to switch the actuation methodwhile maintaining smoothness of a displayed image.

FIG. 8 is a flowchart showing determination processing of an actuationswitching method for suppressing variation in time since the start ofmoving image recording is instructed until the high quality actuationmethod is set in this modification. The same step numbers are assignedto the same processes as those shown in FIG. 7 , and the descriptionthereof is omitted.

If it is determined in step S701 that the current frame rate is nothigh, the process proceeds to step S801. In step S801, it is determinedwhether the time from the timing at which the start of moving imagerecording is instructed to the timing at which the next pulse of the VDsignal is input is equal to or less than a predetermined threshold T2.

If the time is equal to or shorter than the threshold T2, the processproceed to step S702 where it is determined that the second actuationswitching method is to be used to switch the actuation method, and ifthe time is longer than the threshold T2, the process proceed to stepS703 where it is determined that the first actuation switching method isto be used to switch the actuation method. Here, it is conceivable thatthe threshold T2 may be set according to a time taken to switch theactuation method in the first actuation switching method, for example.In the first actuation switching method, there is a maximum of 16 ms ofSync synchronization period before issuing the next pulse of the VDsignal in accordance with the display timing of the display unit 311, sothe threshold T2 may be 17 ms which is the sum of the 16 ms of Syncsynchronization period and the setting prohibition period of 1 ms.

FIG. 9 is a timing chart showing an example of switching operation of anactuation method performed when the start of moving image recording isinstructed in a case where the frame rate is low (here, 15 fps). If thestart of moving image recording is instructed before 17 ms or less fromthe next pulse of the VD signal, it is determined that the secondactuation switching method is to be used to switch the actuation method.If the start of moving image recording is instructed at other timings,it is determined that the first actuation switching method is to be usedto switch the actuation method.

By operating as described above, it is possible to further suppressvariation in the time taken for switching the actuation method whilemaintaining smoothness of a displayed image at the time of switching theactuation method as much as possible.

Note that the present invention is not limited to the above-describedembodiments, and can be applied to cases where the actuation method isswitched when the LV mode is switched to another LV mode, and theactuation method is switched when the moving image shooting mode isswitched to another moving image shooting mode.

Second Embodiment

Next, a second embodiment of the present invention will be described.

In the first embodiment, switching of actuation method of the imagesensor 306 when the LV mode is switched to the moving image shootingmode has been described. On the other hand, in the second embodiment,switching of actuation method of the image sensor 306 at the time ofswitching from the LV mode to the still image shooting mode will bedescribed.

Note that the configuration of the image capturing apparatus in thesecond embodiment is the same as that described with reference to FIGS.1 to 3 in the first embodiment, so the description thereof is omittedhere.

First Actuation Switching Method

FIG. 10 is a timing chart showing the switching operation for the imagesensor 306 by the first actuation switching method in the secondembodiment. It should be noted that the case of using the secondactuation switching method for switching from the still image shootingmode to the LV mode is shown.

In the following description, the image sensor 306 is actuated by a lowimage quality actuation method in the LV mode, and the image sensor 306is actuated by a high image quality actuation method in the still imageshooting mode. Further, it is assumed that the frame rate of the imagesensor 306 is 30 fps (33 ms) in the LV mode, and 60 fps (16 ms) in thestill image shooting mode. Furthermore, it is assumed that time taken toread out the signal from the image sensor 306 in the LV mode or stillimage shooting mode is 2 ms. It is also assumed that the display unit311 operates at 30 fps in the second embodiment.

Furthermore, in the following description, since the contents of controlperformed in response to the SIO communication signal 121 are different,the SIO communication signal 121 is described as SIO (5) to (11)communications according to the contents of control in order todistinguish between them.

When the still image shooting is instructed by turning on the switch SW2of the operation unit 315 during the LV mode, the overall controlcalculation unit 309 performs SIO(5) communication to send an actuationswitching request to the image sensor 306. In the SIO(5) communication,the setting such that the image sensor 306 transits to the standby stateat the timing of the next pulse of the VD signal being input isperformed. The timing at which this SIO(5) communication is performedchanges depending on at which timing in the blanking period the stillimage shooting is instructed.

FIG. 10 is a diagram showing a case where a still image shooting isinstructed during the setting prohibition period of 1 ms before the apulse of the VD signal is input, which is a case where it takes thelongest time since the still image shooting is instructed until adisplayed image is changed. In this case, since the SIO(5) communicationis performed immediately after the next pulse of the VD signal isgenerated, the reading of the signal from the image sensor 306corresponding to EXPOSURE (n+1) in FIG. 10 is stopped in the middle, andan image that should be DISPLAY (n+1) cannot be captured. As a result,an image DISPLAY (n), is continuously displayed on the display unit 311for two or more frames, which causes a phenomenon in which the displayedimage looks temporally stopped.

When the image sensor 306 enters the standby state according to theinput of the next pulse of the VD signal through the SIO(5)communication, the overall control calculation unit 309 performs SIO(6)communication during the standby state. The SIO (6) communication is tochange the actuation method of the image sensor 306, send the VD signaland the HD0 signal to the image sensor 306 in accordance with thechanged actuation method, set an exposure period via the reset signallines 104, and further set gains to column ADC blocks 111.

Next, the overall control calculation unit 309 issues the next pulse ofthe VD signal in synchronization with the display timing of the displayunit 311 (Sync synchronization period), the image sensor 306 is changedto the accumulation state, and an exposure for the still image shootingis started. Since the sync synchronization period is between 3 ms and 16ms, it takes about 17 ms at the longest from when the actuationswitching request is input to when the exposure for the still imageshooting is started.

Next, when charge accumulation for a still image is completed, a pulseof the VD signal is issued and SIO(7) communication is performed. In theSIO(7) communication, the reading of the signal from the image sensor306 is set to be performed by the still image shooting actuation methodand the exposure is set to be performed by the high image qualityactuation method at the timing of the next pulse of the VD signal. Ifthe HD period (horizontal period) differs between the still imageshooting actuation method and the high image quality actuation method,the HD0 signal is used in the still image shooting actuation method, andthe HD1 signal is used in the high image quality actuation method tocontrol the actuation timing of the image sensor 306. In addition, theVD signal for the high image quality actuation method is sent to theimage sensor 306, an exposure period for the high image qualityactuation method is set via the reset signal lines 104, and gains for astill image are set in the column ADC blocks 111.

Next, when a pulse of the VD signal is generated 16 ms later, SIO(8)communication is performed. In the SIO(8) communication, the reading ofthe signal from the image sensor 306 is set to be performed by the highquality actuation method and the exposure is set to be performed by thehigh image quality actuation method at the timing of the next pulse ofthe VD signal. Further, an exposure period for the high image qualityactuation method is set via the reset signal lines 104, and gains forthe high image quality actuation method are set in the column ADC blocks111.

Since the Sync synchronization period is between 3 ms and 16 ms, in thiscase, it takes about 37 ms at the shortest and about 50 ms at thelongest from when the still image shooting instruction is input untilwhen an image (DISPLAY (n+2)) obtained next by the high image qualityactuation is displayed in the display unit 311. Note that if the stillimage shooting instruction is input at other timing than the settingprohibition period, the SIO(5) communication is immediately performed,so it takes about 36 ms at the shortest and about 49 ms at the longest.

As described above, in the first actuation switching method, aphenomenon in which the displayed image looks temporally stopped occurs,but the time taken from when the still image shooting instruction isinput until when a displayed image is changed falls within a rangebetween about 36 ms and about 50 ms. Further, as described above, ittakes about 17 ms at the longest from when the still image shootinginstruction is input until an exposure by the still image shootingactuation is started.

Second Actuation Switching Method

FIG. 11 is a timing chart showing the switching operation by theseamless actuation switching method as the second actuation switchingmethod of the image sensor 306 in the second embodiment. Also, a case inwhich the second actuation switching method is used for switching fromthe still image shooting mode to the LV mode is shown.

In FIG. 11 , too, the image sensor 306 is actuated by the low imagequality actuation method in the LV mode, and the image sensor 306 isactuated by the high image quality actuation method in the still imageshooting mode. Further, it is assumed that the frame rate of the imagesensor 306 is 30 fps (33 ms) in the LV mode, and 60 fps (16 ms) in thestill image shooting mode. Furthermore, it is assumed that time taken toread out the signal from the image sensor 306 in the LV mode or stillimage shooting mode is 2 ms. It is also assumed that the display unit311 operates at 30 fps in the second embodiment.

When still image shooting is instructed by the operation unit 315 duringthe LV mode, the overall control calculation unit 309 performs SIO (9)communication as an actuation switching request to the image sensor 306.In the SIO(9) communication, the reading of the signal from the imagesensor 306 is set to be performed by the high image quality actuationmethod and the exposure is set to be performed by the still imageshooting actuation method at the timing of the next pulse of the VDsignal. If the HD period (horizontal period) differs between the highimage quality actuation method and the still image shooting actuationmethod, the HD0 signal is used in the high image quality actuationmethod, and the HD1 signal is used in the still image actuation methodto control the actuation timing of the image sensor 306. In addition,the VD signal for the still image shooting actuation method is sent tothe image sensor 306, an exposure period for the still image actuationmethod is set via the reset signal lines 104, and gains for the highimage quality actuation method are set to the column ADC blocks 111.

Here, in a case where a still image shooting instruction is input attiming 1 in the setting prohibition period of 1 ms before a pulse of theVD signal is input, it takes about 34 ms since the still image shootinginstruction is input until an exposure of the still image shooting isstarted, wherein the 34 ms consists of the setting inhibition period of1 ms, and the 1 VD period of 33 ms.

On the other hand, if the still image shooting instruction is input attiming 2 immediately before the setting prohibition period, it takesabout 1 ms since the still image shooting instruction is input until anexposure of the still image shooting is started, which corresponds to 1ms of the setting inhibition period. That is, a variation of 33 msoccurs depending on the timing at which the still image shootinginstruction is input.

Next, when a pulse the VD signal is generated after 16 ms, SIO (10)communication is performed. In the SIO(10) communication, the imagesensor 306 is controlled such that the still image shooting actuationmethod will be set for readout and the high image quality actuationmethod will be set for exposure at the next pulse of the VD signal. Inaddition, the VD signal for the high quality image actuation method issent to the image sensor 306, an exposure period for the high qualityimage actuation method is set via the reset signal lines 104, and gainsfor the still image shooting actuation method are set to the column ADCblocks 111.

Next, when a pulse of the VD signal is generated after 16 ms, SIO (11)communication is performed. In the SIO(11) communication, the imagesensor 306 is controlled such that the high quality actuation methodwill be set for readout and the high image quality actuation method willbe set for exposure at the next pulse of the VD signal. In addition, anexposure period for the high quality image actuation method is set viathe reset signal lines 104, and gains for the high quality imageactuation method are set to the column ADC blocks 111.

Switching Control of Actuation Method

In the second embodiment, too, as shown in the flowchart of FIG. 7 inthe first embodiment, in a case where the frame rate before switchingthe actuation method is low, the first actuation switching method, notthe second actuation switching method, is used, thereby suppressing thevariation in time from when the still image shooting instruction isinput to when switching to the still image shooting actuation method isperformed. However, in the example of the second embodiment, in thedetermination in step S701, for example, a frame rate of 60 fps or moreis determined to be the high frame rate.

It should be noted that the switching method may be determined as shownin the flowchart of FIG. 8 in the modification described above. In thatcase, the threshold T2 may be set to 17 ms, which is the longest time inthe first actuation switching method, for example.

As described above, according to the second embodiment, a variation intime taken to switch the actuation method can be suppressed whilemaintaining smoothness of a displayed image as good as possible at thetime of switching actuation method from the LV mode to the still imageshooting mode.

Note that the present invention is not limited to the above-describedembodiments, and can be applied to cases where the actuation method isswitched from the moving image shooting mode to the still image shootingmode, and the actuation method is switched from the still image shootingmode to the moving image shooting mode.

Third Embodiment

Next, a third embodiment of the present invention will be described.

In the third embodiment, another method for determining which of thefirst actuation switching method and the second actuation switchingmethod should be used at the time of switching the actuation method ofthe image sensor 306 will be described.

Note that the configuration of the image capturing apparatus in thethird embodiment is the same as that described with reference to FIGS. 1to 3 in the first embodiment, so the description thereof is omittedhere.

FIG. 12 is a flowchart which shows the processing for determining whichof the first actuation switching method and the second actuationswitching method is to be used in a case where an instruction involvingswitching of the actuation method of the image sensor 306, such as aninstruction to start moving image recording or a still image shootinginstruction, is input according to the third embodiment.

Here, for example, in a case of displaying an image using an EVF, theimage sensor 306 is actuated by ⅓ thinning readout in the verticaldirection, and in a case of displaying an image using an LCD, the imagesensor 306 is actuated by ⅕ thinning readout in the vertical direction.Thus, in a case of switching the display unit 311 between the EVF andthe LCD according to an operation to the operation unit 315, theactuation method of the image sensor 306 is also switched.

When the determination processing of the actuation switching method isstarted, in step S1101, it is determined whether or not the switching ofthe actuation method of the image sensor 306 accompanies the switchingbetween the EVF and the LCD of the display unit 311.

If it is determined that the switching of the actuation method of theimage sensor 306 is due to the switching between the EVF and the LCD,the process proceeds to step S1102 and it is determined that the firstactuation switching method is to be used to switch the actuation method.If it is determined in step S1101 that the switching of the image sensor306 is not due to switching between the EVF and the LCD, the processproceeds to step S1103.

In step S1103, it is determined whether the switching of the actuationmethod of the image sensor 306 is due to switching from a moving imageshooting mode to another moving image shooting mode. For example, themoving image shooting mode with 4K recording size (i.e., a shooting modeof reading out the signals from all pixels and generating a moving imageof 4K recording size) is changed to the moving image shooting mode withthe FHD recording size (i.e., a shooting mode of reading out the signalsby ½ thinning in the vertical direction and generating a moving image ofthe FHD recording size).

If it is determined in step S1103 that the switching of the actuationmethod of the image sensor 306 is due to switching from a moving imageshooting mode to another moving image shooting mode, the processproceeds to step S1108 to perform the switching of the actuation methodby the second actuation switching method. If it is determined in stepS1103 that the switching of the actuation method of the image sensor 306is not due to switching from a moving image shooting mode to anothermoving image shooting mode, the process proceeds to step S1104.

In step S1104, it is determined whether or not the switching of theactuation method of the image sensor 306 is due to switching to ashooting mode in which an image is not displayed. The shooting mode inwhich no image is displayed is, for example, a shooting mode in whichsignals are read out from pixels by 1/32 thinning in the verticaldirection in order to detect flicker.

If it is determined in step S1104 that the switching of the actuationmethod of the image sensor 306 is due to the switching to the imageshooting mode in which the image is not displayed, the process proceedsto step S1102, and it is determined that the first actuation switchingmethod is to be used to switch the actuation method. If it is determinedin step S1104 that the switching of the actuation method of the imagesensor 306 is not due to the switching to the image shooting mode inwhich the image is not displayed, the process proceeds to step S1105.

In step S1105, it is determined whether or not switching of theactuation method of the image sensor 306 is due to switching to ashooting mode that accompanies a change in angle of view. The shootingmode that accompanies a change in angle of view is, for example, ashooting mode in which image data is read out by cropping the signal inthe vertical direction so as to enlarge the image.

If it is determined in step S1105 that the switching of the actuationmethod of the image sensor 306 is due to switching to the shooting modethat accompanies a change in angle of view, the process proceeds to stepS1102, and it is determined that the first actuation switching method isto be used to switch the actuation method. If it is determined in stepS1105 that the switching of the actuation method of the image sensor 306is due to switching to the shooting mode that accompanies a change inangle of view, the process proceeds to step S1106.

In step S1106, it is determined whether or not switching of theactuation method of the image sensor 306 is due to a change in subjectluminance, in such a case as the subject becoming bright. If it isdetermined in step S1106 that the switching of the actuation method ofthe image sensor 306 is not due the subject becoming bright, the processproceeds to step S1102, it is determined that the first actuationswitching method is to be used to switch the actuation method. If it isdetermined in step S1106 that the switching of the actuation method ofthe image sensor 306 is due to the subject becoming bright, the processproceeds to step S1107.

In step S1107, it is determined whether or not the timing at which anactuation method switching request is issued is within a period of apredetermined threshold value T2 before the next pulse of the VD signalis generated. If it is determined in step S1107 that the timing at whichthe actuation method switching request for switching the actuationmethod of the image sensor 306 is issued is within the period of thethreshold value T2 before the next pulse of the VD signal is generated,the process proceeds to step S1108, and it is determined that the secondactuation switching method is to be used to switch the actuation method.

If it is determined in step S1107 that timing at which the actuationmethod switching request for switching the actuation method of the imagesensor 306 is issued is not within the period of the threshold value T2before the next pulse of the VD signal is generated, the processproceeds to step S1102, and it is determined that the first actuationswitching method is to be used to switch the actuation method.

As described above, a variation in time taken to switch the actuationmethod can be suppressed while maintaining smoothness of a displayedimage as good as possible at the time of switching the actuation methodby changing between first actuation switching method and the secondactuation switching method.

OTHER EMBODIMENTS

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2021-191441, filed Nov. 25, 2021 which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An electronic device comprising: an image sensor;and one or more processors and/or circuitry which function as: an inputunit used for inputting an instruction that accompanies switching of anactuation method of the image sensor; and a controller that switches theactuation method of the image sensor using either of a first switchingmethod and a second switching method based on a predetermined conditionaccording to an input of the instruction, wherein, in the firstswitching method, the actuation method is switched via a standby stateof the image sensor according to the input of the instruction, andwherein, in the second switching method, the actuation method isswitched at a timing of a start of a next frame immediately after theinput of the instruction while actuating the image sensor at apredetermined frame rate.
 2. The electronic device according to claim 1,wherein in a case where the frame rate before switching the actuationmethod of the image sensor is less than a predetermined first threshold,the controller switches the actuation method using the first switchingmethod, and in a case where the frame rate before switching theactuation method of the image sensor is equal to or greater than thefirst threshold, the controller switches the actuation method using thesecond switching method.
 3. The electronic device according to claim 1,wherein in a case where the frame rate before switching the actuationmethod of the image sensor is less than a predetermined first thresholdand an input timing of the instruction is before a period of apredetermined second threshold from the start of the next frame, thecontroller switches the actuation method using the first switchingmethod, and in the other cases, the controller switches the actuationmethod using the second switching method.
 4. The electronic deviceaccording to claim 1, wherein the instruction includes an instruction torecord a moving image issued during a live view image is displayed, andan instruction to perform still image shooting issued during a live viewimage is displayed.
 5. The electronic device according to claim 1further comprising a plurality of display units, wherein the controllerswitches the actuation method using the first switching method in a casewhere at least one of a plurality of conditions is met, wherein theplurality of conditions includes a case in which the instruction is tochange a display unit used to display an image, a case in which asetting is such that no image is displayed on the display units at thetime of switching the actuation method, a case in which the instructionaccompanies a change in angle of view, and a case in which it isdetermined that brightness of a subject is brighter than a predeterminedbrightness.
 6. The electronic device according to claim 5, wherein, in acase where none of the conditions is met, the controller switches theactuation method using the second switching method.
 7. The electronicdevice according to claim 1, wherein in a case where brightness of asubject is brighter than a predetermined brightness and an input timingof the instruction is within a period of a predetermined secondthreshold from the start of the next frame, the controller switches theactuation method using the second switching method, and in a case wherethe brightness of the subject is not brighter than the predeterminedbrightness or an input timing of the instruction is before the period ofthe second threshold from the start of the next frame, the controllerswitches the actuation method using the first switching method.
 8. Theelectronic device according to claim 1, wherein, in a case of, whileperforming moving image recording, switching to another moving imagerecording with a different resolution, the controller switches theactuation method using the second switching method.
 9. A control methodof an electronic device comprising: switching, according to an input ofan instruction that accompanies switching of an actuation method of animage sensor, the actuation method of the image sensor using either of afirst switching method and a second switching method based on apredetermined condition, wherein, in the first switching method, theactuation method is switched via a standby state of the image sensoraccording to the input of the instruction, and wherein, in the secondswitching method, the actuation method is switched at a timing of astart of a next frame immediately after the input of the instructionwhile actuating the image sensor at a predetermined frame rate.
 10. Anon-transitory computer-readable storage medium, the storage mediumstoring a program that is executable by the computer, wherein theprogram includes program code for causing the computer to perform acontrol method of an electronic device comprising: switching, accordingto an input of an instruction that accompanies switching of an actuationmethod of an image sensor, the actuation method of the image sensorusing either of a first switching method and a second switching methodbased on a predetermined condition, wherein, in the first switchingmethod, the actuation method is switched via a standby state of theimage sensor according to the input of the instruction, and wherein, inthe second switching method, the actuation method is switched at atiming of a start of a next frame immediately after the input of theinstruction while actuating the image sensor at a predetermined framerate.